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Abstract 

This 10 week longitudinal evaluation study evaluated a brain-based 
learning curriculum proposed by Smilkstein (2003) by comparing 
student performance in a traditional basic writing curriculum with 
NHLP-oriented basic writing curriculum. The study included two 
classes each of experimental and traditional methods. Results of 
the data, gathered by means of questionnaires and in-class writing, 
indicate the experimental classes expressed more positive comments 
than the traditional classes, and, on an enjoyment scale, tended to 
score higher mean Likert scores; but low N-size inhibited statistical 
testing and weakens the claim. However, scores for the final writing 
were statistically significantly higher for experimental students. 


Connecting theory to practice is considered a helpful and necessary 
component of successful developmental education programs (Boylan, 
2002; Casazza, 2003; Casazza & Silverman, 1996; Chung, 2005). Theory 
not only provides an explanation for practice but enables reflection and 
thus development (Griffee & Gorsuch, 1999). However, as Saxon and 
Boylan (2003) point out, theories need to be empirically investigated to 
determine which hold promise. Brain-based learning theory and its resulting 
curriculum, as discussed by Smilkstein (2003), provide one theory currently 
of interest. 


Background 

Smilkstein (2003) referred to her work in learning theory as Natural 
Human Learning Process (NHLP), which included five assumptions. First, the 
brain is conceived as an organ that actively constructs knowledge. Second, 
teaching means giving students the opportunity to actively learn. Third, if 
students fail to comprehend and apply knowledge, it is because they need 
additional background and preparation, not because they lack the ability to 
understand the material. Fourth, the pedagogical sequence is "the teacher 
should first give students opportunities to be active learners; then the 
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teacher, if necessary, can add to what the students have discovered on their 
own by giving a lecture, definitions, background, technical terms, 
explanations, examples, demonstrations, and so forth" (Smilkstein, p. 4). 
Fifth, learning is pleasant when students have had a chance to actively learn, 
as compared to a more traditional understanding that asserts learning is 
pleasant when the teacher is pleasant and the material is entertaining. 

Working within the tradition of second language acquisition, Ellis (1998) 
discussed task evaluation and proposed a three-part evaluation model: 
student-based, response-based, and learning-based evaluation. Student- 
based evaluation gathers data about student opinion, response-based 
evaluation gathers data on the extent learning took place with students 
doing exercises under direct instructor supervision, and learning-based 
evaluation gathers data on the extent to which learning took place without 
direct instructor supervision. This evaluation model provides multiple 
evaluation paths that can result in a more complete view than any single 
view can provide. 

Research Questions 

The purpose of this 10-week longitudinal evaluation study was to evaluate 
a brain-based learning curriculum by comparing the performance of students 
in a traditional basic writing curriculum with those in an NHLP-oriented basic 
writing curriculum. Three research questions were developed, one each from 
student-based evaluation, response-based evaluation, and learning-based 
evaluation: 

1. From a student-based perspective, did students in the NFILP classes 
find the curriculum more enjoyable than students participating in the 
traditional curriculum? If Smilkstein is correct that learning is pleasant when 
students have a chance to learn actively, this study should indicate students 
in the NHLP classes expressed more enjoyment than by students in the 
traditional classes. 

2. From a response-based perspective, will students from NHLP classes 
complete their writing assignments and receive higher cumulative scores 
than students in traditional classes? If students are actively involved in their 
own learning and writing, this study should reflect those students will take 
more responsibility for their assignments, complete them on time, and thus 
obtain higher total scores than students in a traditional writing curriculum. 

3. From a learning-based perspective, does an NHLP curriculum result in 
equal or better student writing scores than scores from a more traditional 
curriculum over one semester? If students in a NHLP-oriented curriculum 
are actively constructing knowledge, the resulting writing scores would be at 
least equal or, perhaps, surpass the writing scores of students in a more 
traditional curriculum. 

Description of Smilkstein Model as used in this study 

According to Lalicker (2001), traditional basic writing curriculum is defined 
as a below college-level, not full-credit course for writing at the paragraph 
level designed to prepare students for academic discourse, typically using 
lectures and grammar exercises. According to Smilkstein (2003), NHLP basic 
writing curriculum is defined as the same as the traditional curriculum with 
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the following modifications: All writing is evaluated using a midterm 
assessment and is introduced using an NHLP writing workshop format. The 
NHLP format used in this study consisted of four parts, with each part having 
three stages. 

In part one, students worked individually and wrote down what they did 
during the 30 minutes before class. In the second stage, students worked 
in pairs or small groups and read their narratives to each other. The third 
stage consisted of looking at the similarities and arriving at a definition of a 
narrative with the instructor writing all contributions on the board without 
comment. 

In part two, students again worked individually and wrote from when 
class started to 30 minutes before class started. For example, students 
would write sentences similar to "I sat down in my seat; just before that, I 
came into the room." In the second stage, students worked in small groups 
and read their narratives to each other; they were encouraged to change 
their definition if they thought it could be improved. Students were also 
asked to identify words or phrases they wrote that showed movement in 
time going forward or backward. Finally, with the instructor writing the 
transition words on the board, students worked as a whole class to report 
changes in their definition of narrative and what words or phrases they used 
to show movement in time. 

In part three, students were given a handout of an example narrative text 
by the instructor and asked to individually to write notes about the author's 
time sequence and the use of transitions. In stage two, students worded in 
small groups and shared discussed their notes. In stage three, students 
working as a whole class were asked to write their findings on the board. The 
instructor lectured points as seemed appropriate. 

In part four, students were assigned as homework to write a narrative of 
their own choice based on their experiences using transitions to help readers 
follow the movement through time backwards and/or forwards. 

Method 


Participants 

I taught four basic writing courses used in this research. Two were 
randomly selected for the NFILP curriculum and two for the traditional 
curriculum. Classes were selected so as to balance day of week and time of 
day. The study began with a total of 37 students, 18 in the traditional classes 
and 19 in the NFILP classes, and ended week 10 with a total of 23 students, 
10 in the traditional classes and 13 in the NFILP classes. The average age of 
all students was 21; the youngest was 17 and the oldest was 29. Students 
in the classes using the traditional curriculum were one African-American, 
five Caucasian, one Hispanic, and three international students from Thailand, 
Viet Nam, and Cameroon. Students in the classes using the NHLP curriculum 
were one African-American, six Caucasian, two Hispanic, and two international 
students, one from Korea and one from China. 
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Instruments 

To answer research question one, an enjoyment questionnaire was 
designed consisting of one open-ended item and one closed-ended item. 
The open-ended item was "In terms of what we actually do in class, what do 
you enjoy or not enjoy about our class?" The close-ended item was a Likert 
scale of one to six in which one and two were designated as not fun, three 
and four were designated as so-so, and five and six were designated as 
enjoyable. The item was stated as "Circle the number that best shows overall 
how much you enjoy this class." The questionnaire was administered after 
the completion of each of the four writing modules: narrative, descriptive, 
comparison and contrast, and persuasive. Students were instructed not to 
sign their name, but to fill out the form and put it on a chair near the door 
as they left the room. 

To answer research question two, points and due dates were designated 
for all assignments, and a late policy was published in the syllabus that 
stated assignments could be handed in no more than one class late. Each 
student was given a point sheet, and points were entered as writing was 
received on a duplicate point sheet kept in the class folder. Reflective letters, 
response papers, brainstorms, writing plans, and first drafts were given full 
point credit if they were completed on time, but final drafts were graded 
by the instructor. Scores thus reflect to a large extent the degree to which 
students submitted material on time. 

To answer research question three, a prompt was given to all students 
requiring an in-class, persuasive essay of at least one paragraph. 

Data Collection and Analysis 

The enjoyment questionnaires were collected, enjoy versus not enjoy 
comments were identified, and the number of positive comments were 
summed and divided by the number of students making the comments, 
resulting in mean frequency scores. Comments not relevantto the curriculum, 
such as "I don't enjoy writing," were not tabulated. Likert ratings were 
summed and divided by the number of students answering, which provided 
an average for each class. Students' points earned were entered into a 
computer spreadsheet on the last class day of each writing module, which 
resulted in four collection periods. Point totals were individually summed, 
and a class average was calculated. 

In-class essays were collected and graded by the instructor using the 
grading criteria for persuasive writing as seen in Figure 1. To deal with 
instructor rater bias, all students were instructed to sign their composition 
on the back. In addition, four essays, one each from an African-American, a 
Caucasian, a Hispanic, and an international student were randomly selected 
from the NHLP classes, and three essays, one each from an African-American, 
a Hispanic, and an international student, were selected from the traditional 
classes. After rater training was conducted, these seven essays (three from 
men and four from women) were given to a second faculty rater to grade. 
Instructor scores and second rater scores were correlated, corrected for 
attenuation according to a formula from Brown (1996, p. 155), and reported 
as a reliability coefficient. This reliability coefficient demonstrates the degree 
to which the course instructor rated students in the traditional and NHLP 
curriculum classes consistently. 
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Grading standards for the final persuasive writing 

1. Writing is within the genre. This includes four features: 

♦ A topic sentence that clearly states the proposition and writer's 
stance toward it e.g., I think the city should pass an ordnance 
prohibiting smoking in public, 

♦ reasons which are clearly and logically related to the proposition, 

♦ evidence which supports the reason, 

♦ and a conclusion. 

2. Maintains paragraph unity. The paragraph must be coherent. 
For example, does the writing contain transitional phrases 
(e.g., first, second, next, on the other hand). 

3. Has audience awareness. Audience awareness means the 
text is reader based, not writer based. Reader based text is 
from the reader's point of view which means the reader can 
understand the meaning. Writer based text is text which makes 
sense to the writer, but not to the reader. An example of writer 
based text is, "In conclusion the reason that this lesson is so 
important to me, is for, with in this one lesson there are many 
more to come." 

4. Appropriate vocabulary. The vocabulary is appropriate for 
academic use, and is not speech-based that contains slang or 
conversational phrases. 

5. Mechanics. Mechanics refers to fragments, run-on sentences, 
comma splices, or misspelled words. 

Grading scale: 

It's all there = 20 points. 

It's mostly there = 15 points. 

Some is there = 10 points. 

A little is there = 5 points. 

Nothing to grade = 0 points. 

Figure 1. Grading standards for the final persuasive writing 
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Results 

Research question one was addressed by the frequency of positive and 
negative comments as shown in Table 1 and the Likert scale ratings as 
shown in Table 2. As can be seen in Table 1, the NHLP classes began by 
making fewer positive comments than students in the traditional classes 
(3.80 to 6.00). After week 5, both groups seem about equal, but by the end 
of week 7, the NHLP classes were making more positive comments. This 
trend continued through week 10. 


Table 1 

Average Frequency of Positive and Negative Comments 


Collection period 

NHEP 

positive 

1 raditional 
positive 

NHUP 

negative 

1 raditional 
negative 

End of week 3 


iroo 

T94 

T33 

End of week 5 

2.66 

2.83 

1.00 

1.33 

End of week 7 

2.00 

1.34 

0.94 

0.93 

End of week 10 

1.43 

1.00 

0.25 

0.13 

The Likert scale enjoyment 

ratings can be seen in Table 2. Both the 

NHLP classes and the traditional class scores start at about the same point, 

4.59 and 4.78. 

At the end of week 5, the NHLP classes enjoyment ratings 

increased to 5.08 while the traditional 

class enjoyment scores decreased 

slightly to 4.71 

. By the end of week 7 

, both groups 

were the same, but 

by week 10, some difference reappeared. Of interest 

is the final standard 

deviation for both groups. Since standard deviation is a measure of how the 

scores group around the mean (Vogt, 

1999), the standard deviations for 

week 10 indicate a higher level of agreement among students in the NHLP 

classes than in the traditional classes. 



Table 2 





Likert Scale Enjoyment Ratings 




Collection period 

NHLP classes 


l raditional classes 


M 

SD 

M 

SD 

End of week 3 

T55 

TJ2 

T7B 

.67 

End of week 5 

5.08 

.90 

4.71 

.95 

End of week 7 

4.50 

1.24 

4.50 

.93 

End of week 10 

5.00 

.43 

4.50 

1.38 


Research question two was addressed by students' earned points collected 
at the end of weeks 3, 5, 7 and 10, which can be seen in Table 3. These 
weeks were chosen because they were the final weeks of the four writing 
modules. The NHLP classes both scored writing score totals in the mid- 
four hundreds. One of the traditional classes also scored in the mid-four 
hundreds and one did not. This evidence is inconclusive and may or may not 
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indicate that a NHLP oriented curriculum motivates students to turn in their 
writing assignments. 

Table 3 

Cumulative Student Earned Points 


Classes/Weeks 

Week Three 

Week Five 

Week Seven 

Week Ten 

NHLP Classes 

MWF 9:00 am 

80.1 

195.0 

313.4 

469.2 

TT 11:00am 

65.3 

199.0 

294.0 

455.0 

Traditional Classes 

MWF 10:00am 

53.9 

215.0 

331.0 

479.0 

TT 9:30am 

73.5 

154.0 

261.0 

321.0 


Research question three was addressed by the scores from the final essay 
as seen in Table 4. Rater reliability corrected for attenuation was .90, 
indicating high consistency between the scoring of the instructor and the 
second rater, which indicates that the instructor who rated the essays from 
all students was rating the essays written by the students in the control and 
NHLP classes in a similar way. The NHLP mean score was 82.92, and the 
traditional mean score was 71.61. 


Table 4 

Final Writing Average Scores 



NHLP 

Traditional 

N 

13 

10 

Mean 

82.69 

72.00 

SD 

8.81 

13.98 

Minimum 

70 

50 

Maximum 

100 

90 

Skewness 

.17 

-.37 

Kurtosis 

-.64 

-.78 


After verifying that all assumptions for a t-test had been met, results of a 
one-tailed t-test were t = 2.246, df = 21, p = .0178. Strength of association 
calculated using Cohen's d was .77, indicating a fairly robust finding. 

Discussion 

Research question one was, "Will students in the NHLP classes find the 
curriculum more enjoyable than students in classes using the traditional 
curriculum?" The answer is a tentative yes, although the evidence is not 
conclusive. The experimental NHLP classes expressed more positive 
comments than the traditional classes, and on an enjoyment scale using 
Likert scale ratings, they also tended to score higher mean Likert scores 
than the traditional classes, but because of low number of students involved, 
no statistical comparisons were done which weakens the claim. 
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Research question two was, "Will students from NHLP classes complete 
their writing assignments in a way so to obtain scores higher than students 
in traditional classes?" Again, the answer is yes; two of the NHLP classes 
scored higher than one of the classes in the traditional curriculum. It must 
be pointed out, however, that one of the classes in the traditional curriculum 
also scored a high number of points indicating that more variables may be 
involved than this paper investigated. Nevertheless, these findings may be 
important because it suggests that an NHLP curriculum encourages students 
to complete and hand in their work which is a prerequisite for student writing 
improvement. 

Research question three was, "Does an NHLP curriculum result in equal or 
better student writing scores than a more traditional curriculum?" The 
answer is yes; the NHLP classes scored an average of ten points higher than 
the classes receiving the traditional lecture curriculum. To put it another 
way, the average score obtained on the final writing by the students 
experiencing the traditional curriculum was a C while the average score 
obtained by students experiencing the NHLP curriculum was a B. This average 
score difference occurred despite a drop-out rate that left only determined 
students in both the control and experimental classes, which resulted in a 
high level of one-on-one teacher to student instruction in both control and 
experimental classes. 

What factors in the NHLP curriculum promoted more student enjoyment, 
generated higher performance, and increased learning? After reviewing 
field notes kept during the semester, two reasons suggest themselves: the 
workshop methodology and the midterm formative evaluation. There was 
no difference between the control and NHLP groups with reference to the 
teacher, the way students were selected for the course, the day or time 
of the class, the types of writing, the method of grading, or practice in 
mechanical drills. The workshop methodology, on the other hand, was a 
major difference. 

Each workshop began with an individual student task that provided data 
for the workshop. Stage one (individual) allowed every student to participate 
in a concrete rather than abstract way. Considerable student resistance at 
stage one (whole group discussion) occurred in one class when asked for 
their definition of a narrative. Students wanted a "correct answer" and 
insisted that they did not know the definition of a narrative piece of writing. 
This impasse provided an instance of teacher-student interaction. The 
instructor insisted on an answer until one or two students offered a few 
phrases toward a definition. Phrases were written on the board without 
comment. At stage two (the whole class), additional comments were added 
to the definition that made it more complete. 

This study suggests the bottom-up recursive stages of the workshop 
promoted teacher-student interaction that allowed students to construct 
their own understanding of the writing process. This happened to a far lesser 
degree in the traditional curriculum classes, which rely primarily on lectures 
and present students with lectures resulting in minimal interaction. 

The second major difference between the control and experimental groups 
was the midterm evaluation conducted at the end of week seven. Smilkstein 
(2003) gave detailed step-by-step procedures for what she calls "student 
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group instructional feedback," (p. 164). This study, however, is using the 
more traditional term: Midterm Evaluation (ME). 

A teacher colleague agreed to come into the classroom and conduct the 
ME. Procedures were established, and a complete class session was allocated 
to the ME. The colleague came into the classroom, was warmly greeted to 
make it clear that the instructor was not being evaluated, and then the 
instructor left the room. The colleague posed three questions: what works, 
what doesn't work, and what could be improved? Students wrote their 
answers individually, shared in small groups, and then the answers were 
written on the board. A student volunteer transcribed all the answers, and 
the colleague led a whole class discussion and asked if there was agreement 
on all the points under the three questions. This last point ensures that the 
answers are a class consensus rather than a collection of idiosyncratic points. 
The colleague collected the transcriptions and a day later gave them to the 
instructor along with her understanding of what happened. 

In the class immediately following the midterm evaluation, the results 
were discussed and clarification sought of the issues raised by the transcripted 
answers. It was found that what worked was the brainstorming and writing 
plan assignments. What did not work can be grouped under four categories: 
student misunderstandings, unacceptable requests, idiosyncratic points, 
and requests concerning instruction and feedback. 

An example of student misunderstanding occurred in their request for 
more interesting writing topics. This was a misunderstanding because the 
instructor never assigned writing topics but only agreed to provide students 
with topics if they could not come up with a topic themselves. Elimination of 
peer evaluation represented an unacceptable request. One idiosyncratic 
request slipped through the midterm evaluation. In one class, students 
requested more grammar lessons, but when quizzed on this point, it was 
found that while one student made the request and the other students 
agreed during the midterm evaluations the class later acknowledged they 
did not agree although they said so at the time. A final suggestion was a 
request for more feedback from the instructor and more direct instruction on 
writing. 

For the remainder of the course, the number of required drafts was 
increased from two to three. Draft one underwent peer evaluation and 
received full points if turned in. Draft two was evaluated by the instructor 
and would also receive full points. Draft three was graded by the instructor 
based on the criteria in Figure 1. These changes met the concerns of the 
students, increased feedback from the instructor to all students, and allowed 
for more direct instruction. 

Implications and Further Research 

As a result of this study, I am no longer willing to present writing genre in 
lecture format. Flowever, in recent graduate level classes which include ESL 
students, there have been requests for lectures, and I may have to decide 
time, form, and content of classes. Perhaps the role of lecturing and its place 
in classroom instruction is more complex than first imagined and future 
research could help. 
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The evidence here suggests lecture fronted instruction is not as helpful as 
NHLP instruction. Should lectures be eliminated in favor of various forms of 
class discussion? Smilkstein (2003) herself indicates that there might be a 
place for lectures after students have had a chance to engage in active 
learning. This suggests that it is the active learning which prepares students 
to more fully grasp the lecture content. Reversing the format of lecture- 
discussion to discussion-lecture could be a future research area. 

Another research area for future research is the function and role of mid- 
term evaluation (more traditionally referred to as formative evaluation). 
As the results of this study show, the trend of all evaluation indexes was 
down until formative evaluation and the steps suggested by that evaluation 
were taken. Formative evaluation takes into account that teachers have 
blind spots and cannot see certain areas of student concern. This study 
strongly suggests that student input and feedback can make a valuable 
contribution. 


Conclusion 

In summary, the midterm evaluation produced formative evaluation data 
that allowed an "interactional" dialogue between the instructor and the 
classes and, in turn, resulted in curriculum changes. The midterm evaluation 
apparently had a rejuvenating effect on the NHLP classes as reflected in the 
Likert scale approval ratings in Table 2 and the cumulative score differences 
in Table 3. In both cases, after week seven, scores reversed their downward 
direction and rebounded substantially. 

This empirical evaluation study comparing a brain-based writing 
curriculum based on Smilkstein (2003) with a traditional writing curriculum 
produced evidence to suggest that a brain-based curriculum is promising 
to composition teachers. Despite a very high drop out rate (typical for 
developmental education courses at this institution), the data from this study 
provides evidence that over a period of time, a NHLP curriculum provided 
higher student approval ratings, increased participation, and improved 
student writing scores. 
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